When a user searches for images on a network such as the Internet the user initiates a query for images, often via textual input (e.g., typing the keywords “Porsche 911”). In response to the query, a search is performed, and images similar to the query are returned as a set of image search results (also known as results or query results). Traditionally image search results are found and ranked based upon matching the keywords with text associated with images (e.g., the HTML code, text adjacent to the image on a web page, image captions, etc.) on a network or images on a network that have been indexed. However, such methods are limited. For instance, synonyms may not be captured by such a search. Images appearing without text, such as automatically generated security camera images, may not be found in a search. Hence, in terms of relevance and diversity, queries that match text can only go so far.
Some methods try to overcome these limitations by extracting a predefined set of features from images (e.g., color, texture, edge, shape, etc.) and comparing these features to the user's textual query or keywords or to images selected by the user. In this way, the images themselves rather than just text associated with the images are analyzed. Content Based Image Retrieval (CBIR) is one example of such image analysis. CBIR is also known as query by image content (QBIC) and content-based visual information retrieval (CBVIR). These methods entail extracting content, data, or features (e.g., color, edge, texture) from images and using them to rank those images relative to a query. For example, color can be extracted for each image by plotting a histogram of the number of pixels of ranges of colors. As another example, texture can be extracted via identifying patterns of pixels within an image and comparing these to patterns in other images. Another example is shape, where edge detection identifies shapes within an image and compares these to identified shapes within other images.
For further information regarding the state of the art see, Content-based Multimedia Information Retrieval State of the Art and Challenges Michael Lew et al., ACM Transactions on Multimedia Computing, Communications, and Applications, pp. 1-19, 2006.